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Fidelity is a word that is often used but rarely understood when talking about ground- 
based simulation. Assessing the cueing fidelity of a ground based flight simulator requires a 
comparison to actual flight data either directly or indirectly. Two experiments were 
conducted at the Vertical Motion Simulator using the GenHel UH-60A Black Hawk 
helicopter math model that was directly compared to flight data. Prior to the experiment 
the simulator’s motion and visual system frequency responses were measured, the aircraft 
math model was adjusted to account for the simulator motion system delays, and the motion 
system gains and washouts were tuned for the individual tasks. The tuned motion system 
fidelity was then assessed against the modified Sinacori criteria. The first experiments 
showed similar handling qualities ratings (HQRs) to actual flight for a bob-up and sidestep 
maneuvers. The second experiment showed equivalent HQRs between flight and simulation 
for the ADS33 slalom maneuver for the two pilot participants. The ADS33 vertical 
maneuver HQRs were mixed with one pilot rating the flight and simulation the same while 
the second pilot rated the simulation worse. In addition to recording HQRs on the second 
experiment, an experimental Simulation Fidelity Rating (SFR) scale developed by the 
University of Liverpool was tested for applicability to engineering simulators. A discussion 
of the SFR scale for use on the Vertical Motion Simulator is included in this paper. 


I. Introduction 

I t’s impossible to provide the same cues in a ground based simulator as actual flight but can ground based 
simulators provide sufficient cues to obtain comparable pilot performance? Research and development (R&D) 
simulators like the Vertical Motion Simulator (VMS) at NASA Ames Research Center typically judge simulator 
fidelity by comparing handling qualities ratings (HQRs) between simulator and flight for a given task. It has been 
argued by White et. al. that matching HQRs is not a good indicator of simulator fidelity for training purposes 1 . 
White believes a better way to measure fidelity is compare piloting technique between the aircraft and simulator for 
a given task. Regardless of how simulator fidelity is measured, obtaining adequate fidelity can be challenging due to 
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necessary compromises required for ground based motion simulation such as motion/visual system transport delays, 
reduced motion envelope, visual cueing differences, and math model fidelity. 

Considerable research has been completed on various aspects of ground based motion simulation to better 
understand and mitigate the adverse effects of the simulation system on pilot performance. Knotts and Bailey 
advocated keeping the added delay from the simulator motion below 50 msec so the pilot does not differ their 
control strategy from flight 2 . Mitchell et. al. showed that a motion transport of 80 msec would degrade the handling 
qualities rating (HQR) from Level 1 to Level 2. 3 Sinacori hypothesized, 4 and later Schroeder extended, 5 a criterion 
for defining the quality of simulator motion based on the gain and phase of the motion software filters. Mitchell and 
Hart suggested minimizing the mismatch between motion and visual delays. 6 Gum and Martin suggested some 
techniques to reduce math model delays. 7 All the research listed addresses individual aspects of ground based flight 
simulation, but does not look at the simulation system as a whole and compare it to actual flight data. 

In 2012 two experiments were run on the VMS that assessed the fidelity of a UH60A Black Hawk helicopter 
simulation. The first experiment, named SimOpt, reproduced a bob-up and sidestep task from a previous experiment 
run on the VMS in conjunction with flight testing in 1989 and 1990. 8 The second experiment evaluated the new 
Simulation Fidelity Rating (SFR) scale developed at the University of Liverpool for use on a R&D simulator. 9 A 
slalom and vertical maneuver were performed in a UH60 Blackhawk helicopter and repeated the in the VMS. 

These experiments, demonstrates the improvement in simulation results when the end-to-end simulation system 
response is optimized to be similar to flight. This paper describes the SimOpt and SFR experiment, the simulation 
systems optimization, and compares the HQR results from the actual flights, 1989-90 simulations, and the SimOpt 
and SFRE simulations. In addition, a discussion of the SFR rating scale for use on an R&D simulator will be 
presented. 


II. Objectives and Approach 


A. Experiment 1 - SimOpt 

In 1989 flight tests were performed with a UH-60A Black Hawk helicopter flying bob-up, sidestep, and 
dash/quickstop tasks at the NASA Ames flight-test facility at Crows Landing Naval Auxiliary Air Station. 
Concurrent with the flight-testing, the same tasks were performed on the VMS using the same four pilots flying the 
UH-60A GenHel math model. During the 1989 simulation a problem with the data acquisition software resulted in 
the loss of all performance data leaving only the subjective data. The simulation was repeated in 1990 using three 
pilots with only two of the pilots that participated in the original experiment. 10 

In the 2012 SimOpt experiment, the bob-up and sidestep tasks were repeated on the VMS while maintaining 
similarity with the 1989 and 1990 simulations. Exact replication of the simulation cueing experiment from the 1989 
and 1990 experiments was not possible due to the upgraded visual database, image system, and motion system’s 
dynamic performance. Various VMS subsystems, such as the computer image generator and sound system, have 
been upgraded. In addition, the motion-base frequency response has been improved since 1990. The ICAB used in 
the 1990 simulation could not be used for this experiment because it now has a rear projected visual display system 
and is no longer a collimated system as in 1990. 

The objective of this experiment was to compare pilot- vehicle performance of the bob-up and sidestep 
maneuvers to that of the 1989 and 1990 simulations and actual flight. 

B. Experiment 2 - SFR Testing 

In August 2012 flight tests were performed with a UH-60 Helicopter flying a slalom and vertical task at NASA 
Ames Research Center. Concurrent with the flight-testing, the same tasks were performed on the VMS using the 
same two pilots flying the UH-60A GenHel math model. 

The objective of this experiment was to compare pilot-vehicle performance, in the form of HQRs, for the slalom 
and vertical tasks to that of actual flight. In addition to comparing HQRs, the new Simulation Fidelity Rating scale, 
developed at the University of Liverpool, was tested for applicability to engineering simulators for evaluating 
simulator fidelity. 
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III. Experimental Setup 


A. UH-60A GenHel Math Model 

The GenHel math model configured for the UH-60A helicopter is a nonlinear representation of a single main 
rotor helicopter, accurate for a full range of angles of attack, sideslip, and rotor inflow. It is a blade element model 
where total rotor forces and moments are calculated by summing the forces from blade elements on each blade, 
which are determined from aerodynamic, 
inertial, and gravitational components. 

Aerodynamic forces are computed from 
aerodynamic function tables developed from 
wind tunnel test data. 

Due to the inherent motion system delay of 
the VMS, the GenHel math model was 
modified by removing delay in specific areas to 
provide a more accurate pilot input to motion 
cue representation of the UH-60A vehicle. 

Using the math model outputs as the truth set representing the actual flight vehicle, two techniques to reduce the 
equivalent time delay of the GenHel math model were implemented and tested. The first concentrated on the 
primary servo (actuator) models, and the second focused on the blade-element model of the main rotor. The 
equivalent time delay recovered in the model for the two techniques is show in Table 1 . 

B. Model Configurations 

7. Slalom, and Vertical Configurations 

The baseline configuration is the standard UH-60A GenHel model that was used in the 1989 and 1990 
experiments. There were no modifications to remove excess time delay from the math model to compensate for 
motion system delay (see Table 2). 

2. Bob-up Configuration 

The modified configuration for the bob-up maneuver reduces the time delay in the actuators only and does not 
use the rotor looping technique. The rotor looping technique was not used in the bob-up and vertical maneuvers 
since they are primarily vertical tasks and there is no benefit in the vertical axis (see Table 2). 

3. Sidestep Configuration 

The modified configuration for the hover and sidestep maneuver utilized both the actuator and rotor looping 
techniques to reduce the model delay (see Table 2). 


Table 1. Equivalent time delay recovered from model. 


Axis 

Actuator Only 
(sec) 

Actuator and Rotor 
Looping (sec) 

Pitch 

0.016 

0.064 

Roll 

0.013 

0.045 

Yaw 

0.014 

0.005 

Vertical 

0.014 

0.016 


Table 2. Equivalent Time Delay of Simulation System by Maneuver. 


Axis 

Slalom & Vertical Maneuver 

Bob-up Maneuver 

Sidestep Maneuver 

Baseline GenHel (sec) 

Modified GenHel 
(Actuators only) 
(sec) 

Modified GenHel 
(Actuators plus rotor looping) 
(sec) 

Pitch 

0.047 

0.034 

0.002 

Roll 

0.068 

0.052 

0.004 

Yaw 

0.048 

0.034 

0.032 

Longitudinal 

0.050 

0.050 

0.050 

Lateral 

0.069 

0.069 

0.069 

Vertical 

0.067 

0.053 

0.062 
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IV. Results 


A. Experiment 1 - SimOpt 

The HQRs maximum, average and minimum values are shown for each category denoted by the vertical bar 
showing the range of the minimum and maximum value with the solid square representing the average. 


1. Bob -up Maneuver 

The average HQR is less than the 1990 simulation but 
slightly more than the 1989 simulation (see Fig. 1). The 
possible reason for the SimOpt experiment HQR 
improvement over the 1990 experiment is the VMS 
motion system was tuned specifically for the task, and the 
motion system performance has been improved over 
time. The average flight test HQR is less than one rating 
point better than the SimOpt configurations. The pilots 
on average were able to achieve Level 1 handling 
qualities for the SimOpt configuration. 

It is difficult to compare flight HQRs with simulation 
HQRs unless the maneuvers are evaluated back-to-back 
with the same pilots. The HQRs from the 1989 
simulation were performed in conjunction with the test 
flights showed the best correlation. The 1990 simulation 
that was conducted six months after the flight test shows 
a significant increase in the average HQR as compared to 
the 1989 simulation. Atencio 9 states, “It appears as if the 
flight experience was enhanced with passing time and 
unfavorable flight characteristics were forgotten.” as a 
possible reason for the worse HQRs. The HQRs from 
both SimOpt configurations are similar to the 1989 
simulation though different pilots participated and there 
was no concurrent flight test. 

2. Sidestep Maneuver 

The average HQR for the modified GenHel 
configurations HQRs are similar to those from 1989 and 
1990. The flight test average HQR from the flight test is 
less than a point better that the modified GenHel 
configuration. 
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Figure 1. Bob-up maneuver HQRs. 
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Figure 2. Sidestep maneuver HQRs. 


B. Experiment 2 - SFR 

7. Slalom Maneuver 

The HQRs for both pilots were nearly identical between flight and simulation (see Fig. 3). The SFRs for both 
pilots were in Level 2 of the SFR scale (see Fig. 4). The SFR ratings did not change between the both pilots first run 
and when they became proficient. Both pilots were unable to fly the task as fast in the simulator and had greater 
variations in altitude (see Fig. 5 and 6). 
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Figure 3: Slalom Maneuver HQRs. 
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Figure 4: Slalom Maneuver SFRs. 
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Figure 5: Slalom Position for Pilot 1 
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Figure 6: Slalom Ground Speed and Altitude for Pilot 1 


2. Vertical Maneuver 

The HQR for Pilot 1 were identical between flight and simulation. The HQR rating for Pilot 2 was three 
points higher in the simulator than in flight (see Fig. 7). The SFR for Pilot 1 was four after the first run but 
improved to two after becoming proficient at the task. The SFR for Pilot 2 was nine after the first run but was 
improved to seven after becoming proficient (see Fig. 8). 
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Figure 7: Vertical Maneuver HQRs. 
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Figure 8: Vertical Maneuver SFRs. 


The performance of Pilot 1 in the simulation was similar to that of flight. The collective inputs and position 
excursions were similar between flight and simulation (see Fig. 9 through 11). 
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Figure 9: Vertical Maneuver Position for Pilot 1 



Figure 10: Vertical Maneuver Altitude for Pilot 1 
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The performance of Pilot 2 on the first run in the simulation was different than flight. The simulation 
performance of Pilot 2 became similar to flight after becoming proficient at the maneuver (see Fig. 12 through 14). 
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Figure 12: Vertical Maneuver Position for Pilot 2 
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Figure 13: Vertical Maneuver Altitude for Pilot 2 



Figure 14: Vertical Maneuver Collective Position for Pilot 2 


There are several possible explanations for the difference in HQR and SFR ratings between Pilot 1 and Pilot 2. 
The VMS is an engineering simulator and not intended for training so the cockpit layout is not the same as the actual 
aircraft. Pilot 2 had little experience in the VMS and was not familiar with the cockpit layout as compared to Pilot 1 
who has extensive VMS experience. Pilot 2 flew the actual aircraft from the left seat and was unable to see the 
hover board used for longitudinal cues, so Pilot 1 informed Pilot 2 of his positional drift from the right seat. In the 
simulation Pilot 2 was in the right seat and was required to determine his own longitudinal position. These cockpit 
and task differences were most likely reasons for the differences in SFRs and HQRs for Pilot 2. 
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